fix(pricing): generate the embedded price table from the live feed - #13
Merged
Merged
Conversation
The embedded table answers every lookup the override and remote layers do not: UseRemote off, network down, or a first refresh still in flight. Hand-maintained and dated "as of April 2026", it had drifted. gemini-2.5-flash carried an output rate of 0.60 against a real 2.50 — under-reporting spend on that model by more than 4x, in the direction that silently defeats budget caps. The Claude 5 and 4.6 families were missing outright, so an offline caller running them got nothing at all. defaultPrices now lives in defaults_gen.go, produced by pricing/gen from the LiteLLM feed. Rather than reimplement rate selection, the generator installs a real snapshot and reads it back through pricing.Lookup, so vendor precedence, dropped rows and cache backfill apply exactly as they do at runtime and the two cannot diverge. Regenerating uncovered two further defects, both fixed here. Canonical() reduced "anthropic.claude-v1" and "claude-v2:1" to a bare "claude" at Claude 1's 8.00/24.00. longestPrefix matches at segment boundaries, so that key then answered for every Claude model not listed explicitly: claude-haiku-3-5 resolved to 8.00/24.00 against its real 0.80/4.00, a 10x over-report. An ID that has lost all version information is a family name, not a model, so the qualified form is kept instead. This also stops the legacy entries collapsing together, which is why the snapshot now holds 1086 models rather than 1070. The curated list needed claude-haiku-3-5 sourced from upstream's claude-3-5-haiku — Anthropic put the generation before the tier on the 3.x line and after it from 4.x on. The generator now requires an exact key in the snapshot rather than accepting whatever Lookup's prefix fallback returns, so a curated ID that has quietly gone missing fails the build instead of inheriting a sibling's price. Rates are rounded to ten decimal places on the way out. Upstream quotes per-token, so every value arrives multiplied by a million and 2e-07 renders as 0.19999999999999998, which is unreadable and makes each regeneration a noisy diff. A weekly workflow regenerates and fails if the committed table has moved, which is the signal that a vendor changed a price. The generator leaves the file untouched when only its own timestamp would change, so that job fires on real drift rather than every Monday.
|
Warning Review limit reached
Next review available in: 10 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (5)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Reviewer's GuideThis PR replaces the hand-maintained embedded pricing table with a generated snapshot from the live LiteLLM feed, fixes a canonicalization bug that over-reported Claude family prices, adds provenance for compiled-in prices, and introduces tests and CI to keep the defaults table correct and current. Sequence diagram for the new pricing generator using the live LiteLLM feedsequenceDiagram
participant Gen as gen_main
participant Pricing as pricing_pkg
participant LiteLLM as LiteLLMSource
Gen->>Pricing: UseRemote(ctx, RemoteConfig{Sources: [LiteLLMSource], RefreshInterval: -1})
Note over Pricing,LiteLLM: LiteLLMSource installs snapshot into remote layer
Gen->>Pricing: AsOf()
Pricing-->>Gen: asOf, source
Gen->>Pricing: KnownModels()
Pricing-->>Gen: []modelID
loop curated models
Gen->>Gen: source() // choose id or alias from
Gen->>Gen: present[modelID]
alt [modelID present]
Gen->>Pricing: Lookup(modelID)
Pricing-->>Gen: Price, ok
alt [ok]
Gen->>Gen: prices[id] = Price
else [!ok]
Gen->>Gen: missing = append(missing, modelID)
end
else [modelID missing]
Gen->>Gen: missing = append(missing, modelID)
end
end
alt [missing not empty]
Gen->>Gen: return error
else [all present]
Gen->>Gen: render(prices, asOf, source)
Gen->>Gen: sameRates(old, new)
alt [rates unchanged]
Gen->>Gen: leave defaults_gen.go
else [rates changed]
Gen->>Gen: write defaults_gen.go
end
end
Flow diagram for updated Canonical model ID normalizationflowchart TD
A[Input model ID] --> B[stripGatewayPrefix]
B --> C[stripRegionPrefix]
C --> D[stripPrefix]
D --> E[stripProviderPrefix]
E --> F[lowercase]
F --> G[strip versionSuffix]
G --> H[strip dateSuffix]
H --> I{contains any digit?}
I -->|yes| J[return unqualified ID]
I -->|no| K[return qualified ID]
subgraph Legend
direction LR
L1[qualified = ID after provider/gateway/region stripping]
L2[unqualified = ID after version/date stripping]
end
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The embedded table answers every lookup the override and remote layers don't:
UseRemoteoff, network down, or a first refresh still in flight. Hand-maintained and dated "as of April 2026", it had drifted:gemini-2.5-flashgpt-4oUnder-reporting is the dangerous direction: it silently defeats budget caps and biases quality-per-dollar toward whichever model's price is stalest.
What
defaultPricesmoves todefaults_gen.go, generated bypricing/genfrom the LiteLLM feed viago generate ./pricing.The generator doesn't reimplement rate selection — it installs a real snapshot and reads it back through
pricing.Lookup, so vendor precedence (#12), dropped rows and cache backfill apply exactly as at runtime. The embedded table cannot diverge from the rules the remote layer follows.Two more defects this uncovered
A 10x over-report.
Canonical()reducedanthropic.claude-v1andclaude-v2:1to a bareclaudeat Claude 1's 8.00/24.00.longestPrefixmatches at segment boundaries, so that key answered for every Claude model not listed explicitly —claude-haiku-3-5resolved to 8.00/24.00 against its real 0.80/4.00. An ID that has lost all version information is a family name, not a model, so the qualified form is kept. (This also stops the legacy entries collapsing together: the snapshot now holds 1086 models, up from 1070.)A silent prefix fallback in the generator.
claude-haiku-3-5isn't an upstream ID — Anthropic put the generation before the tier on the 3.x line (claude-3-5-haiku) and after it from 4.x on.Lookuphappily prefix-matched it. The generator now requires an exact key in the snapshot, so a curated ID that quietly goes missing fails the build rather than inheriting a sibling's price; an explicitalias()keeps llmgate's public ID stable.Staying current
A weekly workflow regenerates and fails if the committed table moved — the signal that a vendor changed a price. The generator leaves the file alone when only its own timestamp would change, so the job fires on real drift rather than every Monday. Verified:
go generatetwice in a row leavesgit diffclean.Tests
TestFamilyNameIsNotAModel— the 8.00/24.00 regression.TestGeneratedDefaultsAreSane— ten rates pinned to vendor list prices.TestNoAbsurdDefaults— units-error guard across every known model, including "a cache hit is never dearer than an uncached token".TestDefaultsAsOfIsParseable— the newDefaultsAsOf()provenance stamp.Full suite green locally.
Closes HAL-564
Summary by Sourcery
Generate the compiled-in pricing defaults from the live LiteLLM feed and harden pricing provenance and model ID handling.
New Features:
Bug Fixes:
Enhancements:
CI: